General Disclaimer 


One or more of the Following Statements may affect this Document 


• This document has been reproduced from the best copy furnished by the 
organizational source. It is being released in the interest of making available as 
much information as possible. 


• This document may contain data, which exceeds the sheet parameters. It was 
furnished in this condition by the organizational source and is the best copy 
available. 


• This document may contain tone-on-tone or color graphs, charts and/or pictures, 
which have been reproduced in black and white. 


• This document is paginated as submitted by the original source. 


• Portions of this document are not fully legible due to the historical nature of some 
of the material. However, it is the best reproduction available from the original 
submission. 


Produced by the NASA Center for Aerospace Information (CASI) 



CK. /S'/r/JL 





soeNoe 

AfDplKXrtlONS 

INCX)RPORaleD 

(NASA-CR-151 812} HAN-MACHINF INTERFACE 
ANALYSIS OF THE FLIGHT DESIGN SYSTEM 
(Science Applications, Inc.) 37 p 
HC A03/HF A01 CSCL 05H 

G3/54 


/ 

N78-29754 


Unclas 

28501 



MAN-MACHINE INTERFACE ANALYSIS 
OF THE 

FLIGHT DESIGN SYSTEM 


by 

H. Rudy Ramsey 
Michael E. Atwood 
John K. Willoughby 


Technical Report No, 
SAI -78-089-DEN 


30 June 1978 



Science Applications, Inc. 


40 Denver Technological Center West, 7935 East Prentice Avenue, Englewood, Colorado 80111. 303/773*6900 


other SA) OfNcee: Albuquerque. Ann Arbor, AHtngton. Atlanta. Soaten. Chicago, Huntsville. La JOiia. lot Angeles. MeUean, Palo Alto, Santa Barbara. Sunnyvale, and Tucson. 



FOREWORD 


This document is the final report of work performed in the 
period 1 March 1978 - 30 June 1978 on Contract No. NAS9-15535 (SAI 
Project No. 1-032-00-129), entitled, "Man-Machine Interface Analysis 
of the Flight Design System." The purpose of this project was to con- 
duct a brief, broad human factors analysis of the Flight Design System, 
a systen intended for use in shuttle-era flight design by the Mission 
Planning and Analysis Division, NASA Johnson Space Center. The human 
factors analysis was intended to provide specific recommendations wher- 
ever appropriate, and to identify potential problem areas involving 
human factors issues. 
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INTRODUCTION 


Flight design for the Shuttle flights of the Space Transportation 
System (STS) Imposes several new requirements on the Mission Planning and 
Analysis Division (MPAD) of the NASA Johnson Space Center. The most 
serious of these requirements Is the high flight rate, which Is projected 
to reach approximately 50 flights a year by 1983 and Is much greater than 
that of previous manned spaceflight prograns. 

Currently, flight planning Is accomplished with large, batch com- 
puter systems. These systems, however, are not sufficient to support the 
high flight rate of the Space Shuttle era. Clearly, a new approach to 
flight design Is required. This approach should be a production-oriented 
system that provides the analyst with automated analytical tools and 
allows for rapid. Interactive flight design. In addition, this system 
should provide an automated documentation process for the production of 
standardized flight profiles and associated documents. This would free 
the analyst from many documentation tasks, which require approximately 
50-60% of the analyst's time under the current system, and would allow 
the analyst to devote more time to actual flight design. 

MPAD Is currently developing such techniques and aids. The Flight 
Design System (FDS) Is intended to provide a powerful, flexible system 
for use In flight design. Experience at JSC during previous manned 
space programs has an^ly demonstrated that the flight design process 
requires the judgment and Intervention of experienced flight planners. 

Yet the level of effort required to support anticipated STS flight rates 
using the flight design techniques of previous programs would be pro- 
hibitive. Thus, the FDS must preserve the critical elements of human 
participation while maximizing the computer's role In the flight design 
process. This efficient combination of human skills and computer 
capabilities requires that the human factors aspects of the FDS must be 
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rigorously evaluated and carefully designed based on established 
principles of effective man-computer interaction. 
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EVOLUTION OF THE FLIGHT DESIGN SYSTEM 


The Flight Design System concept has evolved from the basic com- 
putational techniques already in use in the batch environment of pre-STS 
flight planning. The functional prototype (FDS-1) which has been deve- 
loped uses the basic system architecture of the FDS concept, but contains 
application processors derived, for the most part, directly from existing 
batch tools, especially in the trajectory area. 

It is important to recognize that the purpose of FDS-1 is to 
allow testing of the basic system architecture and functions. The proto- 
type system provides a good, general-purpose capability, but possesses 
only a limited set of computational aids. Furthermore, up to the present 
time, little or no explicit attention has been given to human factors 
issues. This was a conscious decision on the part of MPAD. It was 
generally felt that resources should be concentrated on achievement of 
a working basic system capable of providing an experimental testbed, 
and only then should human factors consultation be sought. Because of 
the very flexible nature of the FDS architecture, modification of fairly 
significant functional aspects of FDS is still possible prior to com- 
pletion of FDS-2, the first production version of the system. It is in 
this context that the human factors analysis reported here was conducted. 
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NATURE AND OBJECTIVES OF THE HUMAN FACTORS ANALYSIS 


Historically, the field of human factors has dealt heavily with 
such areas as controls and displays, psychomotor tasks, and the appli- 
cation of task analysis and similar job-analytic techniques. Even today, 
many people associate the phrase “human factors" with activities of this 
sort. While these activities are still relevant, the last decade has 
seen a iiwch greater emphasis on cognitive tasks, problem-solving aids, 
and analysis of the problem-solving behavior of coin>uter system users. 
This has resulted In part from the Increasing maturity of the study of 
cognitive psychology and human Information processing, and In part from 
the rapidly Increasing use of computers as aids for problem-solving 
tasks, whereas earlier coim)uters were used primarily to support clerical 
tasks. 


As a result of this shifting emphasis, human factors personnel 
have begun to make significant contributions not only In the area of 
"knobs and dials" — the design of keyboards, formatting of displays, 
etc. — but also In the imjch more basic and significant areas con- 
cerned with analysis of user Information requirements, basic functional 
design of the system, detailed dialogue design, and even the overall 
problem-solving procedures of the user, of which the Interactive tool 
Is only a part. 

The objective of the current effort was to perform a broad 
analysis of the human factors Issues Involved In the design of the 
Flight Design System. The analysis was Intended to Include character- 
istics of the system Itself, such as: 

• The basic structure and functional capabilities of FDS 

• User backgrounds, capabilities, and possible modes of use 

• FDS Interactive dialogue, problem-solving aids 

• System data management capabilities 
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and to Include, as well, such system- re la ted matters as: 

• Flight design team structure 

• Roles of technicians 

• User training 

• Methods of evaluating system performance 

From the start. It was understood that the small size of this 
effort would prevent the development of detailed recommendations In 
many of these areas, but It was felt that a rapid, broad Identification 
of the Issues would be the most cost-effective use of the available 
resources. Wherever possible, specific recommendations have been made. 
In other cases, we have Identified the Issues which seem most In^ortant 
and. In some cases, have suggested additional analyses or experiments 
which might provide resolution. 
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SOME BASIC ISSUES 


There are several basic Issues* pertaining to the design and use 
of FDS, that are particularly Important with rei^ect to a human factors 
analysis. For convenience* we have organized these Issues Into six 
basic categories. As will be seen* however* these categories are not 
completely Independent and consideration of an Issue In one category 
may well have Implications for Issues In other categories. 

1. STS planning requirements can be satisfied only through a new approach 
to the problem of flight design. In previous manned spaceflight programs, 
planning was partitioned by mission phase (ascent* on>orb1t* deorbit) and 
each phase was considered Individually. Such an approach requires effect- 
ive Interfaces among the personnel working on each phase. Such Inter- 
faces Impose commml cation and sequencing problems and may tend to pro- 
duce "bottlenecks" Jn the flight design process. Given the high flight 
rate projected for the Space Shuttle era, such bottlenecks must be 
avoided. 

Under the FDS concept, planning tasks will be partitioned pri- 
marily by expected planning difficulty rather than by mission phase. 

This requires flight design analysts to employ a new approach to flight 
design. A related Issue Is the use of a team concept. As planned, a 
flight design team will consist of 6-9 personnel, possibly from differ- 
ent disciplines* and will Include a team leader who will function as 
data base manager and have primary responsibility for approving flight 
designs. Less complex flights may be designed by teams of only one or 
two members. This concept may require more user versatility than 
previous approaches to flight design. In addition, the allocation of 
tasks to team manbers* documentation, communication paths, etc. affect, 
and are affected by, the structure, or organization, of the team. Some 
structures would likely be more effective than others. These issues are 
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not only Important In their own right, but have strong implications for 
the basic properties and detailed design of FDS, 

MPAD has no experience with the planning mode which will be re- 
quired. If experience and performance data are required to validate, or 
to develop further, the FDS concept, explicit experimentation will be 
necessary. 

2. The users of FDS will have highly varied backgrounds. Users will 
range from highly skilled engineers to technicians. Even among the 
skilled engineers, there will be differences with respect to familiarity 
with interactive systems and there may even be differences in the problem 
solving approaches that are applied to flight design. It is difficult 

to assess the iiT 4 >act of technicians since the training, experience, and 
abilities of technicians are not yet well known. 

How technicians should be trained and used is a particularly im- 
portant issue. This issue has implications for the overall planning 
approach, the organization of flight design teams, and the design of 
FDS. It is Important to consider not only the abilities and training 
of technicians, but also the acceptance of the technician role by 
engineers. 

The training of technicians is of primary importance. Technicians 
must be taught the fundamentals of flight design, but the associated 
physics, mathematics, etc., must be highly simplified. In effect, 
technicians must be presented with an abstract view of flight design. 

Care must be taken to ensure that all relevant aspects are included 
in this abstraction and only inessential aspects are excluded. 

3. Satisfactory planning results, in a high flight rate environment, 
will depend on the development of appropriate planning interfaces with 
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related planning tasks. These tasks Include utilization planning, crew 
activities planning, flight slimjlatlon, etc. Developing effective Inter* 
faces may have Implications fr. the design of FOS. 

Frequently, the conversion to interactive aids, shorter span* 
time, and less labor-intensive planning — beneficial though the trans- 
ition may be — Is accompanied by a more Iterative approach to planning. 
This may be because Iterations are easier to accomplish, or simply be- 
cause errors may easily persist Into a later part of the planning cycle 
before detection. Whatever the cause, this phenomenon can spell disaster 
If the Interfaces among the various stages of ^e planning cycle are 
cumbersome. 

4. The current FOS concept relies hei^vlly on an assumed Independence 
among the analytical steps, which are applied In a sequential, linear 
manner. Such linear planning may be. In some cases, Inconvatible with 
the normal problem-solving behavior of the users. In many kinds of 
planning tasks, problem solving proceeds In a hierarchic, rather than 
linear, manner. Such considerations have Implications for the design 
of the FOS dialogue and also for the Interfaces between application pro- 
cessors. 

5. The success of FOS may well depend on the degree to which it aids 
the analyst In retrieving and recognizing problem-relevant information. 
Such retrieval and recognition often relies on episodic me »ry and other 
cues that are difficult to reproduce or replace in an automated system. 
Examples of episodic memory are recalling that the document you need is 
"the one with coffee stains" or that the necessary formulas are "in the 
book with the green cover." Other relevant cues include abstract 
labelling. The Information, by Itself Is not recognized as i co-tan:; 
rather, the problem solver recognizes the abstract labc'. 

associated with thrt information. The success of FOS nay jt p. 
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development and use of appropriate aids for data labelling* abstractions, 
retrieval, etc. 

6. Satisfactory performance of planning tasks Involving FDS will depend 
heavily on the nature of the relationship between the user and the system. 
Of primary Inportance Is the Issue of user acceptance. In addition to the 
usual aspects, this may also Involve overcoming any perceived threat (loss 
of prestige. Job security, etc.) on the part of flight design personnel. 
This Is best addressed by an appropriate transition from current prac- 
tices to training program to operational use of FDS. 

The success of FDS also depends on an appropriate match between 
FDS functional capabilities, dialogue, etc. and the background and ex- 
periences of FDS users. For exan^le. If different users have basically 
different approaches to problem solving, require different types of 
dialogue support, etc., FDS must acconvnodate these differences. 

The functional capabilities and dialogue of FDS must also match 
the possibly diverse requirements of the various problem-solving tasks 
Involved In planning. For example. If ascent, time-line scheduling, 
reentry, etc., are perceived by users as being different types of problem- 
solving tasks, this has Implications for the design of FDS. 
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FINDINGS 


As currently planned, FDS Is potentially very effective as a 
basic cmnputatlonal aid. Its general-purpose structure Is quite power- 
ful and flexible. Although we have a nuster of reconmendatlons for 
changes In the dialogue, and other properties of the system, very few 
of them are In any way 1nc«npat1b1e with the current basic structure. 

It would appear, however, that the currently-planned FDS system may be 
difficult for computer-naive users to operate. This situation appears 
to be significantly Improvable by the Incorporation of some relatively 
Inexpensive dialogue Improvements, as suggested below. Wherever possi- 
ble, we have tried to provide those solutions which appear to give the 
"most bang for the buck," since they have the greatest probability of 
being accepted and Incorporated. 

In addition to simple dialogue changes, more basic Improvements, 
such as problem-solving aids and tutors, appear quite promising. How- 
ever, the small current effort did not permit a sufficiently detailed 
analysis to develop them In any detail. In most cases, the design of 
such aids requires a more detailed understanding of the users' problem- 
solving practices than new exists for the flight design community, and 
further, study of that problem-solving behavior appears to be the most 
effective next step. In any event, such aids are unlikely to conflict 
with the basic design of the system. 
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Understanding the Analytical Process 


Understanding the problem-solving behavior of the users Is basic 
to the formulation and evaluation of appropriate design aids. We were 
able to conduct a number of Interviews of flight designers In several 
specialty areas (ascent, trajectories In general, attitudes, consumables). 
The designers were asked to describe flight design problems and proced- 
ures, tools (Including even hand calculators), experience with Inter- 
active systems as well as batch systems, and their experience. If any, 
with FDS-1 in particular. The "critica. Incident technique" was also 
employed in an Informal way. In this method, personnel are asked to 
Identify instances of outstanding success or failure (we addressed only 
the latter) of the process (flight design) In which they are Involved. 
Inquiries of this sort often help to Identify particularly weak elements 
of the process which might be Improved through automated aids, better 
'rocedures, etc. Most of the Interviewed designers responded quite 
eadlly to such questions, and some of the responses have a direct 
bearing on FDS design. For example, a Gemini problem was Identified 
which appeared to be due to a failure to update early planning data as 
more exact Information became available. An automated aid, such as 
FDS, might assist with such a problem by means of data dating and auto- 
matic flagging of old data. 

While these techniques were Informative, they are entirely Inade- 
quate for the formation of a detailed understanding of the flight design 
process. Particularly In an abstract area like flight design, few peo- 
ple are able to describe In detail the way in which they solve problems, 
even though they may be quite expert at solving them. Furthermore, 
actual practices and performance often differ from the Individjal's 
perception of what he does. To achieve more exact knowledge of flight 
design behavior, It will be necessary to conduct simulations and ob- 
servation of actual flight planning behavior. We believe that the 


- 11 - 



benefits of such simulations would far outweigh their costs, parti- 
cularly If they are conducted early enough to Impact the tutorial 
and problem-solving aids Incorporated In FDS-2. One of the major 
recommendations of the current study Is that MPAD conduct such Simula 
tions. 


At first glance, the use of FDS-1 as a computational aid In 
such simulations seems quite reasonable. However, It should be noted 
that FOS-1 lacks a con^rehenslve set of confute tional aids, and concen- 
trates primarily In the trajectory area. A more basic difficulty Is 
the fact that FDS does not support the early, abstract portion of the 
planning process at all (see later section on "Abstract Plans"). This 
may, or may not, be a deficiency In terms of automated support of the 
flight design process, but is does make FDS a potentially inappropriate 
tool for gathering data about the whole process. It would appear pre- 
ferable to conduct at least the early simulations using current manual 
(batch-aided) planning methods, and extend the simulations to Involve 
FDS only when that step Is clearly justified. 

Another type of data-gathering effort is also desirable. Re- 
sponses to the critical-incident queslons used In the Interviews 
Indicate that this may be a source of useful Information. Applied In 
the form of a survey, rather than face-to-face Interviews, this approach 
could be Inexpensively applied to the entire user community. With an 
appropriately designed questionnaire, this could be a very cost-effective 
source of Information. 
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Technician Users 


The role in which technicians will be used in STS-era flight 
design is not yet clear, and may vary from a simple clerical aide to 
complete end-to-end planning of simple missions, under engineering 
supervision. Appropriate FDS properties, as well as design procedures, 
depend somewhat on this determination, which depends in turn on a 
determination of technician capabilities, acceptance of technicians by 
engineers, etc. 

At the very least, it is clear that technician users will re- 
quire a more tutorial dialogue than will the experienced engineer. It 
is important to recognize that the technician is new to both flight 
design and the use of interactive computer aids. Thus, the FDS dia- 
logue must not only be sufficiently tutorial to allow mechanical 
operation of the system by relatively inexperienced system users -- a 
provision appropriate even for the engineer users, many of whom are ex- 
perienced only with batch tools — but must also provide flight design 
information in a form comprehensible to users who are relatively inex- 
perienced in the entire area of manned space flight. A system which 
provides extensive guidance with respect to interactive dialogue, but 
little assistance in understanding the meaning of maneuvers, flight 
planning operations, etc., might be usable to engineer users but in- 
adequate for technicians. 

More extensive problem-solving aids might be particularly 
effective for the technician user, who will presumably be performing 
most of his work in accordance with planning procedures which will be 
known and fairly well structured. The difficulty is that they are not 
presently known and well structured. A basic issue here is the select- 
ion of an appropriate problem abstraction for use by the technician. 
Clearly, it will be necessary to provide the technician with a nrare 


- 13 - 



simplified view of the problem than that possessed by the engineer, 
and yet avoid omission of critical elements. For example, the use of 
a graphical analogue, in which the technician relies primarily on 
graphical portrayals of trajectory and attitude Information, Is under 
consideration. While this may very well prove to be an effective 
approach. Its Implementation requires more knowledge of the technician 
user class than we now have. 

MPAD has proposed to conduct an experimental Investigation of 
the use of technicians In flight design. This study would Involve the 
formulation and trial of various approaches to problem abstraction, 
training, etc., with a small number of technicians. We believe that 
this study Is a very important source of Information which will Impact 
not only FDS design (at least as regards tutorial and problem-solving 
aids), but also flight design procedures, team structure, etc. We 
would encourage MPAD to proceed with such a study as soon as possible. 

Abstract Plans 


In many kinds of planning behavior, the planner begins with the 
development of an abstract plan which Is at a relatively high level 
(e.g., "launch, then orbital entry, then ...") and contains little. If 
any, of the detail found In the final concrete plan ("..., then per- 
form 3 fps delta-V maneuver, then ..."). Development from the most 
abstract form may be quite direct, or may involve many steps, each of 
which represents a slightly more concrete statement of one of the 
elements of the hierarchic plan. In flight design, for example, an 
Intermediate planning step might be "perform maneuvers to attain geo- 
synchronous orbit," which Is subsequently broken down into a series of 
specific maneuvers. 
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It Is Important to recognize that FDS currently does not allow 
the formulation, storage, or refinement of an abstract plan. The 
computations with which FDS Is concerned are the most concrete elements 
of the hierarchic plan. The relatlonshipsof these concrete elements to 
the abstract plan are not explicitly represented In FDS. Furthermore, 

It Is possible for FDS application prorissors to cross stage boundaries, 
further complicating the recognition of these relationships. 

The absence of abstract plan Information In FDS Is probably not 
a problem for the experienced engineer. In the case of this user, the 
concrete plan is a familiar representation, and Its ties to abstract 
plans are "overlearned" and obvious. In the case of the technician, or 
even a new engineer with no specific flight design experience, this may 
be a much greater problem. This user is already overloaded with new 
Information, and the waning of "GPMP DELVIO," not to mention the re- 
lationship of this specific computation to the overall flight plan, 
is likely to be less than obvious. Furthermore, experience with plan- 
ning processes of this sort would suggest that the technician user may 
be able to perform satisfactorily In a "schema plus corrections" mode — 
in which an existing basic plan Is selected and modified only in those 
details necessary to make It applicable to a specific mission — but 
that success with this planning mode requires comprehension of the en- 
tire plan and all its elements, abstract and concrete. 

At present, we do not have sufficient data concerning the 
problem-solving behavior of either experienced engineers or technicians 
to determine whether — or how — abstract planning information should 
be incorporated into FDS. At a very specific level, though, it appears 
likely that comments placed directly in the sequence table might assist 
considerably in those instances in which the user must comprehend the 
function of the table. Comments and related features will be discussed 
in more detail in a later section. 
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Seine General Coiranents on FDS Dialogue 


It Is In the area of Interactive dialogue that the current study 
has produced its most specific recommendations. The next few sections 
of this report deal with various aspects of FDS dialogue, but will be 
prefaced with a few general comments which do not clearly fit into the 
later, more specific discussions. 

First, the existing (FDS-1) system requires the transmission, 
from the terminal, of a blank followed by a carriage return to Identify 
a null input. The requirement for a blank character is an extremely 
error-prone feature. Users generally consider the blank to have no 
significance, and it is not displayed. Even though modification of the 
21 MX terminal handler may be required, it is our recommendation that 
this problem be corrected so that a simple carriage return is consist- 
ently recognized as a null input. 

In general, a null input should cause only nondestructuve actions, 
and should usually cause that action which is the most probable need 
of the user (subject to the constraint that the overall pattern of de- 
fault actions must be clear and consistent). There are a few instances 
in the current dialogue in which improvements in default behavior appear 
possible. We would suggest that a link analysis of the dialogue Is the 
best basis for decisions concerning the dialogue default structure. A 
link analysis is a simple tabulation of the frequency with which users 
select each option available at each decision point in the dialogue. 

Such an analysis is relatively inexpensive and can result in noticeable 
improvements in dialogue usability, especially for experienced users. 

In general, destructive actions (such as deletion of the user's 
temporary files) should require at least one explicit user action, and 
preferably two actions, one of which is explicit. In the current FDS 
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design, a user who has built a temporary file with an editor may type 
to return to the FDS executive module. Although no explicit per- 
mission has been given by the user, this action results In Immediate 
deletion of the file. In other cases, Is a normal, nondestructive 
method of returning to the executive. Unexpected destruction of files 
can be costly and extremely Irritating to the user. We would suggest 
that, whenever such Implicit file deletion Is about to occur, an ex- 
plicit Inquiry be made of the user (e.g., "FILE HAS BEEN MODIFIED. OK 
TO DELETEI'O, and that an affirmative response be required ("Y" or "YES") 
before such deletion proceeds. The cost In extra terminal operations Is 
small and Infrequent, but the avoidance of an Inadvertent deletion of 
an extensively modified file can be a significant savings. Of course, 
this procedure should be used only If there is a modified file which 
Is about to be lost. 

Interface of Application Processor to Interface Table 

In the current FDS design, resolution of all Interface table 
values must be accomplished before the corresponding application pro- 
cessor Is called. This has resulted In the frequent adoption of an 
array format for those Interface tables for which a variety of variables 
may be required depending on the option selected (e.g., the General- 
Purpose Maneuver Processor). This Is an undesirable practice, since the 
array name Is not an adequate prompt for the required varlable(s), and 
extended prompts are precluded by this practice. It would appear that 
the most appropriate solution to this problem Is a change In the "bind- 
ing time" of Interface table values, so that Interface table variables 
are only required at the time they are requested by the application 
processor. This would allow the use of explicit variable names, and 
extended prompts, in the Interface table, without necessitating that 
values be provided for variables which are irrelevant for the processor 
option (s) selected. 
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The current FDS design does not allow the user a convenient 
mechanism for ascertaining the relationships between application 
processors and interface tables. It would appear desirable that ab- 
stracts be provided for both of these elements, and that the user 
be able to ascertain the answers to both of the following questions: 
"What application processor is this interface table associated with?" 
and "What interface tables do I have which are associated with this 
application processor?" 

Application Processor Standards 

The usability of a system as complex as FDS is strongly affected 
by the consistency of the conventions used throughout the system. At 
present, most application processors operate in a manner similar to 
batch programs, with the interactive aspects of the dialogue controlled 
almost entirely by the FDS executive and execution processors. As the 
conversion to FDS occurs, however, it is evitable that more interaction 
with the user will occur at the application processor level. This is, 
in fact, the best way to achieve a highly interactive dialogue, where 
it is needed, within the confines of FDS. At present, no standards 
exist which will control the nature or consistency of these interactive 
aspects of application processors. We recommend that such standards 
be established, to include at least the following aspects of application 
processor function: (1) informative comments, (2) error messages, 

(3) basic interactive dialogue conventions, and (4) disposition of inter 
mediate results. The latter concern stems from an instance observed 
in FDS-1 , in which an intermediate result was displayed on the terminal, 
but not otherwise saved, even though it was needed for later pro- 
cessing. 
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Movefnent Among Subsystems 


FDS has several system-level modules (executive, execution 
processor, editors, etc.}. These modules are related to one another 
by means of the system's access structure. That structure Is strictly 
hierarchic, with the executive controlling access to all other modules. 
Thus, a user who wishes to access the Interface table editor from the 
execution processor must first return to the executive. It Is Important 
that such a system access structure satisfy two criteria. First, 1t 
must be sufficiently simple and natural that the user can form a usable 
"mental model" of the system. The current FDS structure appears entirely 
satisfactory on the basis of this criterion. Second, It must be func- 
tionally adequate and usable from the viewpoint of the specific tasks 
which the user must perform. 

While the FDS structure Is functionally adequate. It may be In- 
convenient In some situations. In particular, the user who Is execut- 
ing a sequence table In the semi-automatic mode cannot conveniently 
Invoke the Interface table editor to preview a table. If data are 
missing from the Interface table, the Interface table editor Is Invoked, 
as If It were a subroutine, with automatic return to the calling environ- 
ment upon completion. To accomplish the same function for the purpose of 
a preview, the user must perform four separate access operations (exe- 
cutive, editor, executive, execution processor) and must recreate the 
original environment by providing a sequence table line number. MPAD 
should perhaps consider the adoption of a call -and- return approach to 
editor Invocations In general. 

The use of single, special characters to change system modules, 
and as proim>t characters for the :nodules. Involves a tradeoff. Such 
abbreviation can be very helpful to the highly experienced user, but 
is usually undesirable because Its arbitrary conventions are difficult 
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to learn. In the present case> however, the system structure 1s simple 
and the use of such special characters appears to us to be acceptable. 
Rather than eliminating them, we would suggest the provision of a 
"wordy" mode (see later discussion of "Dialogue Modes") as an appro- 
priate antidote to the difficulty which the new user has with such a 
scheme. 

Command Language 

For those few commands which Involve the specification of irore 
than one parameter (e.g., "SEQEDIT TABLEX, TABLEY"), FDS uses a posi- 
tional notation In which the significance of a particular parameter 
value Is derived from Its position In the parameter list. Positional 
notations are very natural from the viewpoint of the computer proces- 
sing required In their Interpretation, but constitute a significant 
source of transposition errors by the user. (Which Is the old table 
In the above command, and which the new?) Positional notation Is 
minimal In the current FDS design, and Is not particularly objection- 
able In those specific Instances In which It occurs. It Is suggested, 
however, that any tendency for such notation to proliferate, as the 
system coninand language grows, should be carefully considered or avoldf . 
altogether. 

It Is desirable that the particular verbs used In any conmand 
language be easily discrimlnable. Ideally, they should be dlscrlmln- 
able both alphabetically (so that commands can be specified by the 
experienced user with the fewest possible characters) and semantically 
(In the sense that Uie user readily associates each command with the 
function which It performs, rather than confusing two or more commands). 
The FDS command set would benefit from an analysis Intended to Increase 
discrimlnablllty of commands. As an example, the commands "RESTORE" 
and "RECALL" appear to violate both criteria. Complete elimination of 
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such confusabllity Is difficult, and way prove Impossible to attain, but 
considerable liriproveroent should be achievable over the current command 
set. 


When commands have been made maximally dlscrlmlnahla In an al- 
phabetic sense. It Is possible to provide the experienced user with a 
highly abbreviated mechanism for coninand Input. An appropriate mecha- 
nism might, for example, be a multiword, flrst-k-character matching 
algorithm for command recognition. Such an algorithm might allow 

SEQUENCE EDITOR 

to be activated with any of the following abbreviated ccxnmands 

S 

SE 

SEQ 

SED 

SEQEDIT 

SEQUENCE 

With appropriate use of space and comma terminators. It may also be 
possible to stack commands. 

Data Managem ent 

K , w.«ar that the recognition, retrieval, and purging of data 
Is potentially a major problem. Two simple mechanisms seem to hold 
prwnise for helping with very limited aspects of this problem. First, 
data dating, along with automated mechanisms for t>e detection of out- 
of-date data, may help to prevent Inadvertent use of preliminary plan- 
ning data for final mission planning. Second, automated generation of 
abstracts for data elements (see next section) may help to insure that 
these elements are later Identifiable. More basic solutions are un- 
doubtedly needed, but will require more analysis, and more experience 
with FDS, than has been accumulated at present. 
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Coninents and Automatic Abstracting 


For several reasons, already Identified In earlier sections, the 
provision of sequence table comnents appears desirable. Such coninents 
might serve at least three purposes. First, If used appropriately, 
they might greatly Improve the conqirehenslblllty of the sequence table 
Itself. They might Indicate the mission for which the sequence table 
was built. Indicate basic mission phases (the abstract plan) and Indi- 
cate more clearly the function of Individual application processor In- 
vocations. These comments should perhaps be of explicitly different 
types, with an Indentation scheme used to discriminate coninents asso- 
ciated with single sequence table entries from tlwse associated with 
mission phases. 

A second use for sequence table comments Is In the generation 
of appropriate user feedback during sequence table execution In auto- 
matic and semi autcxna tic modes. Combined with appll cation-processor- 
generated comments, these coninents might provide considerable Informa- 
tion to the user If an appropriate “wordy" mode Is selected. As the 
system Is currently configured, the automatic mode user may be asked 
to supply a parameter value for, let us say, the General-Purpose 
Maneuver Processor without any Information about which of a series of 
maneuvers Is Involved. 

A third use for sequence table comments Is in the automated 
generation of abstracts for data elements. Since these files are 
system-generated, they will be Identifiable only if the system or 
the user explicitly adds Identifying Information. An automatically 
generated abstract might contain such Information as the following: 
FLIGHT (taken fran flight comment In sequence table) 

DATE (provided by system) 

USER (provided by system) 
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MISSION PHASE (taken from r»1s:^1on phase conment In 
sequence ta'^le) 

SPECIFIC MISSION STATE (e.C-, "STATE AFTER 10 FPS 

OELTA-V MANEUVER” - obtained by com- 
bination of “STATE AFTFR" with specific 
appl Icatlon-processor-executlon comment 
In sequence table). 

This example else Illustrates the fact that the system may 
iMke speclaMzed use of multiple, explicit comnent types In the 
sequence table. 

Editor Function 


Several specific recommendations appear warranted with respect 
to editor functions. The default mode of operation for the Interface 
table editor Is a ’’fill In the blanks" mode In which the editor auto- 
matically jimtps to the first unresolved variable In the table and 
prompts the user for Its value. This Is probably the appropriate de- 
fault mode for automatic execution of this editor to obtain missing 
values during sequence table execution It probably Is not the appro- 
priate mode If the user overtly r-^ique^its the editor, perhaps for the 
purpose of looking over ths table. In fact, the existing scheme Is 
highly error-prone. The user who enters the editor and attempts to 
list the table Is likely to find (or even not realize) that he has 
typed the value "LIST" for a previously unresolved variable. A similar 
result Is obtained by attempting to escape the “fill In the . ts“ 
mode by a null Input. The blank symbol currently required to .nd cate 
a null Input Is entered as the variable value. These difficulties Pi%y 
be re*.rlvable by use of a normal editor default (pointer at top of 
file, prepared to accept list ccxnnand or explicit variable value sub- 
stitution). It should be noted that an editor pointer, with the 
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capability to list only a few lines of the table, may be required by 
the larger Interface tables which will result from the use of explicit 
variable values. In the "fill 1n the blanks” irode, a blank Input 
should probably be disallowed unless quotation marks are used, and a 
similar treatment of "LIST” may be warranted. 

A very common sequence of user actions Involves replacanent of 
a variable value followed Immediately by the printing of the same line. 
The provision of an automatic "Verify" mode. In which altered lines are 
Immediately redisplayed, would not only reduce the number of user 
actions required, but might also Increase the probability of detection 
of erroneous entries. 

As discussed In a previous section, file deletion on exit from 
the editor should probably require an explicit user action. 

Error Messages 

In general, any error message should Implicitly or explicitly 
convey to the user the following information; 

Source of Message (Executive, application processor, etc.) 
Nature of Error 

Severity of Error and/or Systi-m Action Taken 
User Action Required to Correct the Error 
Thus, a reasonable message might say 

6PMP: NEGATIVE START TIME WAS SPECIFIED. GPMP TERMINATED. 
This message explicitly contains the first three categories of Informa- 
tion, and specifies the nature of the error In sufficient detail to 
clearly Indicate the required corrective action. Few, if any, existing 
FDS error messages satisfy these criteria, and some messages fail to 
satisfy any of the four. The adoption of appropriate error message 
standards, based on the above criteria, is recommended. 
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A second concern with the existing error messages Is the pre* 
sence of unnecessary encoded Information In the text of the messages 
(e.g., “XC02''). While such Information may be Ignored by the experi- 
enced user, It can be quite confusing to the novice. If such Informa- 
tion Is needed by system developers, It should be made available via 
a "Debug" mode, or an extended message replay capability. 

PlaloQue Modes 

Several dialogue mode possibilities should be considered. Dia- 
logue modes allow a one-time-per-sesslon or even one-time-per-user 
selection of options which continue to control the dialogue thereafter 
until the user's election Is changed. 

A "wordy" mode appears to be strongly desirable, and might even 
involve multiple levels. When the wordy mode Is selected, the user 
Is given extensive feedback concerning system function including 
comments generated from sequence tables and by application processors, 
automatic use of extended prompts, etc. This mode is Intended primarily 
for use by novice system users, but may be of continuing utility to 
technician users. 

Several tutorial modes are possible, including: 

A general computer-initiated mode in which all user inputs 
are responses to specific queries by the system. 

An argument query mode In which explicit prompts are made 
for the value of each parameter required or allowed with 
a command (the user must first type the command). 

An abbreviate mode In which the system shows the user the 
most abbreviated way in which he might have specified 
the command he has input. Most systems fail to make 
any provision for teaching the novice user the "tricks" 
available for rapid use. 
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An exp«nd-and-ver1 fy mode In which the system responds to each 
command with a fully expanded statemnt of the consnand, 
Including default values, and awaits user verification that 
the command represents the desired action. This mode 
provides a painless way for the novice to try out his 
expert-user skills. 

A "Debug" mode Is the appropriate mechanism for obtaining extended 
error messages, trace Information, etc., which Is of use only to the 
system developer. 

The use of an editor "Verify" mode has already been discussed. 

User mode profiles, which allow senipermanent selection of all 
desired modes by the user, are a considerable convenience In systans 
with widely varying user preferences or experience. Such profiles 
might be made hierarchic, so that the user may Independently select 
modes or may simply specify the "Beginner" pattern, for example, 

The tutorial Information provided by the system should not only 
be made more extensive, but should probably also be made more context- 
specific. 

Use of Storage-Tube Terminal 

The selection of a storage-tube terminal for FDS imposes severe 
constraints on the Interactive dialogue which the systan can support. 
Computer- initiated command construction (as by menu-selection) Is too 
slow to provide a viable option, and displayed information must be 
accumulated, rather than replaced, whenever update rates are faster 
than a few per minute. These constraints may force revision of the 
existing application processor display philosophy to reduce the amount 
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of Information routinely displayed. 


Determination of the specific way in which this terminal should 
be used (dialogue, display formatting, etc.) requires a detailed anal- 
ysis which Is beyond the scope of the current effort, A few specific 
suggestions can be made, however. First, It will be necessary to have 
the system keep track of the state of each terminal. In the sense of 
the amount of Information currently displayed. The system should stop 
transmitting to the terminal when Its display 1$ full, and await a "go 
ahead" command from the user. It should then Issue a page clear com- 
mand, redisplay the most recent Information, and continue. Although 
the user can manually clear the display, any Information transmitted 
during that operation (which takes 1-2 seconds) Is lost. 

Display windowing, in which the display Is divided Into several 
distinct areas. Is probably the most appropriate general approach to 
display formatting In this situation. In particular, it Is suggested 
that the "run log" Information — In which successive lines typically 
represent successive computational steps — be physically separated 
from the application processor displays, which are typically multiline 
displays. This should reduce confusion of "temporal" and "nontemporal" 
cues. 


Finally, the possibility of locally buffered write-through dis 
plays to enhance the dialogue, should be considered. This feature 
should not be purchased without a detailed understanding of Its In- 
tended use, however, as It alleviates only a few of the constraints 
outlined above. 
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"Audit” Trail 


It Is highly desirable that the system maintain a record of user- 
system interaction at each session. This record should be replayable, and 
thus should be sufficiently detailed to permit a complete regeneration of 
the session. With the simplifying assumption that the systen need not be 
concerned with the initial state of permanent data bases, complete regen- 
eration of the session requires only a record of the user's inputs, which 
are not ordinarily voluminous. On replay, the system should allow mode 
changes (as in turning on "Debug"), single-step execution, restart from an 
intermediate point, etc. 

There are several reasons for our belief that this feature is needed. 
The most basic involves improved communication between the technician and 
the supervising engineer. In the event that the technician attempts to 
carry out the engineer's Instructions, and the result is unsatisfactory, 
the fault may lie with the instructions, the technician's execution of the 
instructions, or the system. In many cases, literal regeneration of the 
session by the technician will be impossible. If the difficulty is to 
be assessed and corrected with minimum thrashing, session regeneration by 
the systen is the indicated mechanism. The engineer can simply step through 
the session with the technician, hopefully recognizing and correcting the 
error. 


A system replay capability is also needed to assist in evaluating user 
complaints. Often, the user encounters an apparent system malfunction, but 
cannot regenerate the error. Often, it is impossible for system developers 
to even determine whether a system or user error is involved, and actual 
isolation and elimination of the system error usually requires that it 
occur in the presence of system personnel. The ability to replay the 
session, especially with trace and extended error message facilities turned 
on, can be quite advantageous. The replay capability can also be useful 
in debugging user programs. 
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Finally, the replay capability can be of immediate use to the user 
who has a "Where am I?" problem, or who needs to return to a previous com- 
putational state in which no "checkpoint" or "save for later use" operation 
was performed. 

Problem-Solving Aids 

In addition to dialogue aids (tutorial features, etc.), which assist 
the user with the mechanical operation of the system, FDS might be made to 
provide guidance with respect to the comprehension or solution of the planning 
problem itself. These aids may range from very simple aids (e.g., the use of 
comments in sequence tables, as already discussed) to very complex and powerful 
decision aids. 

Aside from comments, another simple mechanism which might assist with 
the representation of abstract plans in the system is the appropriate use of 
hierarchic sequence table structure, in which one sequence table is allowed 
to execute another. This feature might allow an overall mission sequence 
table to refer to separate tables for mission phases, etc. This capability 
appears especially desirable if greater provision is made for utility functions 
(initialization, etc.) in FOS-2, as appears to be planned. It is desirable to 
avoid a dominance of the sequence table by commands which are not logically 
important to the understanding of the flight design. 

If technician users are found to be sufficiently capable that they 
become actively involved in sequence table construction , aids for this task 
will probably be needed, and such aids may be useful even for experienced 
engineers. A simple approach to such aids involves construction of the 
sequence table by menu selection (which type of launch do you want, which 
type of entry, maneuvering, etc.?) 
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More powerful problem-solving aids are undoubtedly poss1b1e» and 
may be desirable, but their design should be based on the results of the 
recommended simulation study, the technician study, and more extensive 
experience with the use of FDS-1. Tutorial dialogues could be useful both 
In aiding the skilled analyst In the use of FDS and In aiding the less 
skilled analyst In the flight design process Itself. A related Issue Is 
the use of "critics" or "pre-condition checkers." Precondition checkers 
are used to ensure that the appropriate preconditions for the action the 
user Is currently Initiating are satisfied. These preconditions may 
Include data structures. Interface tables, previous actions that are 
required, etc. Critics are special purpose aids that are Intended to either 
correct or bring to the user's attention errors that frequently occur In the 
flight design process. 

The users of FDS will differ with respect to abilities and previous 
experience. While the highly skilled user may be expected to recognize 
the need for help, either In the use of FDS or In the flight design process, 
this should not be expected of the less skilled (especially the technician) 
user. Techniques that monitor user actions, recognize when the user is In 
trouble, and offer appropriate help could be useful. This may. In part, 
be accomplished by providing for an "audit trail" that records user ana- 
lytical actions and responses (as discussed in a previous section). An 
audit trail would also be useful for determining what information users 
want to retain from one Iteration of a problem to the next and, independ- 
ently of problem-solving aids, would provide Information essential to 
evaluating the use of FDS. 

Subsystem Y Interface 

It may be necessary to make the subsystem X - subsystem Y interface 
bidirectional. In the sense that data can be passed In either direction and 
used for normal computation In the receiving system. This would allow 
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Iterative solutions involving both systems, which may or may not be 
desirable or necessary. It has not yet been demonstrated conclusively that 
such iteration is unnecessary. At a more mundane level, though, the 
ability to pass data from subsystem Y to subsystem X is probably required 
to support the automated production of planning documents. To the degree 
that merging of data from both systems into a single page is required, the 
present concept requires that the merging be performed by FDS (subsystem X). 
Failure to provide a bidirectional interface would require this operation 
to be done manually. 

Transition to FDS Planning 

If MPAO is to fully realize the advantage gained through imple- 
mentation of the FDS-1 prototype system, it is necessary that FDS-1 be 
exercised broadly and formally. An approach which merely makes FDS-1 
available to those who are inclined to play with it is probably inadequate, 
since the personnel who will elect to try it out are not representative 
of the eventual user population. There are both advantages and disadvan- 
tages to an early involvement of those users least experienced with 
interactive aids. Obviously such involvement can result in the recogni- 
tion of system deficiencies which might not otherwise be discovered early 
enough for correction in the normal FDS-2 development effort. On the 
other hand, FDS-1 has intentionally omitted many of the features (tutor- 
ial dialogues, etc.) which might make the system easy for these users to 
learn and operate. Probably a small number of such users should be 
carefully briefed and invited to use the system. In any event, it is 
highly desirable that the evolution toward use of FDS for real production 
planning begin now, in those areas in which FDS-1 provides adequate com- 
putational support. 
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EvaTuation of System Performance 


A proper approach to the evaluation of FDS>aided planning perform- 
ance requires the development of: (1) appropriate problem scenarios, 

(2) good performance criteria, and (3) a performance baseline, probably 
based on manual solution of the same problems. The simulation study 
already recommended for the detailed study of flight designer problem- 
solving behavior would involve development of these materials and data. 

It should, therefore, provide an appropriate basis for performance-based 
evaluations of FDS. It is also appropriate to obtain subjective evaluations 
by the users, using appropriate psychological scaling techniques. 

Flight Design Team Structure 

This is a very complex issue which was not explicitly made a part 
of the statement of work performed here. It must be recognized, however, 
that the structure and role allocation associated with the use of flight 
design teams have implications for the design of the system itself. For 
example, the use of a team approach implies that a given individual may 
be working on several missions in parallel. This in turn suggests the 
possibility that automated planning status displays might be a useful 
adjunct to FDS planning. The use of large teams for relatively difficult 
missions may allow engineers to continue to work in a fairly compartment- 
alized way, while small teams require an ability to use automated planning 
tools outside the user's accustomed area. This has clear implications for 
the system's tutorial and performance feedback features, application pro- 
cessor interfaces, etc. 

Clearly, decisions concerning team structure should be based on an 
understanding of the planning methods employed by flight designers, and 
of the role of technicians in STS flight design, both of which are the 
subjects of possible future studies. These decisions should be deferred. 
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then, but It is also important for MPAD to begin obtaining experience 
with the use of FDS-1 for team problem-solving. This may be a source of 
useful insights with respect to both team structure and FDS design. 

Automated Production of Planning Documents 

The prototype approach which has been developed looks basically 
very good. Two specific features may warrant consideration. First, the 
placement of variable names or labels directly in the master text, rather 
than the use of serial association, would reduce the maintenance effort 
necessary when the text is modified, and might make the master text more 
readable. On the other hand, this feature presents some difficulty with 
arrays and inverted arrays, and renders the spacing of the original text 
different from the final text. Second, the elimination of a requirement 
for exact spacing information in the original text (e.g., this field is 
7 characters long) might also reduce maintenance, assuming that FDS can 
appropriately format the information as it produces the finished document. 
This feature, too, renders the spacing of the original text different from 
the final document. 

Voice-Input Device Study 

By agreement between SAI and MPAD, this issue was not explicitly 
investigated. We did become aware, however, that a similar study is 
currently underway at NASA Ames. 
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